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Abstract 

A  computer  based  medical  diagnostic/information  system  (MEDIC)  which  will 
assist  the  corpsman  in  diagnosis  and  treatment  of  illness  is  being  evaluated 
at  Navy  Hospitals  and  aboard  submarines.  The  system,  an  adaptation  of  one 
developed  by  deDombal  for  use  by  physicians  at  the  University  of  Leeds, 
England,  has  been  successful  in  the  diagnosis  of  abdominal  pain.  The 
diagnostic  program,  based  upon  a  Bayesian  algorithm,  uses  only  information 
obtained  from  patient  history  and  physical  examination.  Corpsmen  are 
given  special  training  in  the  collection  of  this  data.  The  evaluation 
compares  the  accuracy  of  the  diagnostic  information  system,  when  used  by 
trained  corpsmen,  with  the  unaided  diagnoses  of  Navy  emergency  room 
corpsmen  and  physicians.  Use  of  this  system  can  significantly  reduce 
risk  to  the  patient  as  wall  as  unnecessary  and  costly  medical  evacuations. 

An  expanded  version  will  provide  diagnosis,  treatment  guides  and  prognosis. 


The  provision  of  adequate  medical  service  in  isolated  areas  presents 
a  challenge  even  to  the  most  experienced  medical  practitioner.  Sluch 
problems  are  magnified  for  the  Navy  Hospital  Corpsman  who  represents  the 
only  source  of  medical  care  aboard  nuclear  submarines.  While  Corpsmen 
receive  intensive  classroom  medical  training  specifically  designed  to 
prepare  them  for  their  task,  they  must  be  considered  to  be  paramedics  with 
limited  clinical  experience.  Because  he  has  few  medical  facilities 
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available  to  him  and  cannot  communicate  with  other  installations,  the 
difficulty  of  the  corpsman's  task  is  increased.  Such  commonly  accepted 
diagnostic  tools  as  x-ray  and  EKG  are  not  available.  Submarine  crews 
receive  intensive  medical  screening  and  are  a  comparatively  young  segment 
of  the  population.  The  incidence  of  serious  illness  is  low.  However,  the 
occurrence  of  a  single  medical  casualty  which  might  have  been  avoided  is 
not  acceptable.  When  life-threatening  illness  occurs,  the  corpsman  fciay 
request  that  the  patient  be  evacuated  from  the  submarine  to  a  shore  based 
medical  facility.  Such  evacuation  can  result  in  termination  of  the  mission. 
Not  only  is  evacuation  costly  in  terms  of  military  objectives  and  monetary 
involvement,  but  the  transfer  of  the  patient,  who  must  be  removed  by 

\ 

helicopter,  may  result  in  the  loss  of  life,  a  condition  which  it  was 
intended  to  obviate. 

As  a  solution  to  these  medical  problems,  Ityack  and  Moeller  (1976)  have 
suggested  the  development  of  an  interactive  computer  based  medical 
diagnostic/information  system  (MEDIC)  which  would  serve  as  a  surrogate 
medical  consultant.  When  fully  operable,  this  patient  management  system 
would  provide  the  corpsman  with  diagnostic  and  treatment  guides.  Computerized 
diagnostic  tools,  patient  monitoring  devices,  and  refresher  training 
programs  could  be  developed  as  adjuncts  to  the  system  (Ryack,  Henderson,  and 
Moeller,  1979).  A  conceptual  model  of  such  a  system  (Rogers,  Ryack,  and 
Moeller,  1979)  is  shown  in  Figure  1. 

Since  the  most  commonly  reported  illnesses  aboard  nuclear  submarines 
are  those  related  to  abdominal  pain  (Hester,  1971)  and  since  these  patients 
are  those  most  commonly  considered  for  evacuation  (Ryack,  et  al,  1976)  the 


development  of  MEDIC  was  initiated  in  this  area.  From  otxr  survey  of  the 


literature  (Rogers,  et  al,  1979)  an  abdominal  pain  program  developed  by 
deDombal  (1973)  appeared  to  be  most  readily  adaptable  to  our  requirements. 

The  program  could  be  implemented  on  a  mini- computer,  diagnosed  without  the 
use  of  laboratory  tests,  and  was  purported  to  have  a  diagnostic  accuracy  of 
91%  when  used  by  surgeons  (deDombal,  1972,  1973).  With  the  cooperation  of 

Dr.  deDombal  and  the  University  of  Leeds,  this  program  was  adapted  for  use 

C 

on  a  NOVA  1220  computer  and  subjected  to  preliminary  evaluations. 

Subsequently,  it  was  modified  for  use  on  a  TEKTRONIX  4051  desk-top  computer 
(Henderson,  et  al,  1978)  which  is  aboard  nuclear  submarines  and  is  accessible 
to  the  corpsman. 

The  program  utilizes  a  Baypian  model  to  generate  disease  probabilities 

* 

in  eight  areas:  Appendicitis  (APPEND),  Diverticulitis  (DIVERT),  Perforated 
Duodenal  Ulcer  (PERFDU) ,  Cholecystitis  (CHOLE) ,  Small  Bowel  Obstruction  (SBO) , 
Renal  Colic  (R.COLIC) ,  Non-specific  Abdominal  Pain  (NSAP) ,  and'  Dyspepsia  (DYSPP) . 
Included  in  Non-specific  Abdominal  Pain  are  diseases  which  are  not  considered 
to  require  evacuation  such  as  Gastroenteritis,  Urinary  Track'  Infection, 

Cystitis,  and  Mesenteric  Adenitis.  The  structure  of  the  program  is 
summarized  in  Figure  2.  At  the  option  of  the  corpsman,  the  program  will 
generate  data  sheets  (Figures  3  and  4)  which  are  used  to  obtain  the  history 
and  physical  information  required  to  make  a  diagnosis.  After  each  item  has 
been  entered  into  the  computer  a  summary  of  the  data  entered  and  the  diagnostic 
probabilities  for  each  disease  category  are  displayed  for  the  corpsman 
(Figure  5).  A  more  detailed  description  of  the  operation  of  this  program  is 
given  by  Henderson,  et  al,  1978. . 

Although  the  program  has  been  successfuly  vised  by  physicians  at 
Leeds  (deDombal,  1972,  1973),  there  are  differences  between  this  application 
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and  that  of  the  MEDIC  system  which  is  intended  for  use  xn  an  isolated 
non-hospital  environment.  The  Navy  corpsman  differs  from  physicians  in 
training  and  clinical  experience,  and  the  data  base  from  which  the 
diagnoses  is  derived  was  developed  from  a  population  with  characteristics 
different  from  our  population. 

The  present  paper  describes  two  studies  which  were  undertaken-  to 
determine  whether  valid  diagnoses  could  be  obtained  from  the  abdominal 
pain  diagnostic  program  by  Navy  corpsmen  and  to  determine  what  modifications 
might  be  needed  to  increase  the  validity  of  the  program. 

Experiment' 1 

The  purpose  of  the  first  study  was  to  evaluate  the  relative  diagnostic 
accuracy  of  the  computer  when  used  by  corpsmen  in  the  clinical  setting  of  a 
Navy  Hospital.  It  was  conducted  in  the  emergency  room  at  the  Balboa  Naval 
Hospital,  San  Diego,  California. 


Procedure 

Method 

Corpsmen  at  the  Independent  Duty  Technicians  School  (IDT) ,  Health 
Sciences  Education  and  Training  Command,  San  Diego,  and  Emergency  Room 
Physicians  participated  in  the  study.  IDT  students  were  in  the  last  phase 
of  training  during  which  they  rotate  for  two  weeks  through  the  emergency 
room  of  the  Naval  Hospital.  Prior  to  their  rotation,  these  students  were 
given  approximately  eight  hours  of  instruction  in  the  use  of  the  diagnostic 
system.  They  were  instructed  on  how  to  record  and  elicit  the  information 


required  for  completion  of  the  data  sheets  and  in  the  use  of  the  computer. 

The  level  of  training  of  participating  physicians  ranged  from  intern  to 
attending  physician. 

Patients  with  abdominal  pain  previously  undiagnosed  and  lasting  less 
than  seven  days  were  included  in  the  study.  When  possible,  the  IDT- student 
was  the  first  to  evaluate  the  patient.  Either  during  or  after  the  examination 
he  completed  the  data  sheet  on  the  patient.  When  final  disposition  of  the 
patient  had  been  made,  the  data  sheet  information  was  entered  into  tlie 
computer.  Determination  of  final  diagnosis  in  admitted  pat  ents  was  nut’s 
from  discharge  diagnosis,  pathological  diagnosis,  and/or  definitive 
laboratory  tests.  For  those  patients  not  admitted,  with  the  exception  of 
those  cases  where  diagnosis  was  made  by  laboratory  studies,  radiographically, 
etc.,  passage  of  a  12  week  period  v/ithout  a  subsequent  visit  to  the  Naval 
Hospital  established  a  diagnosis  of  "Non-specific  Abdominal  Pain". 

After  final  diagnosis  had  been  determined,  diagnostic  accuracy  of  the 
computer-aided  corpsman  was  established  as  follows :  The  computer-generated 
diagnosis  was  taken  as  any  probability  exceeding  50%.  When  no  disease 
probability  exceeded  50%,  a  diagnosis  of  "Non-specific  Abdominal  Pain"  was 
recorded.  These  computer-generated  diagnoses  were  compared  to  the  final 
diagnoses . 

Results 

One  hundred  eighty-six  cases  of  abdominal  pain  were  obtained  in  the 
period  between  March  1978  and  June  1979.  Of  these,  53  cases  were  omitted 
from  the  evaluation  either  because  the  diagnosis  had  not  been  confirmed. 
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data  was  missing,  or  the  final  diagnosis  was  inappropriate  for  the 
evaluation.  The  overall  diagnostic  accuracy  obtained  was  73.7%  for  the 
physician  and  61.6%  for  the  computer.  Staniland  (1972)  reports  that 
30-40%  of  cases  can  be  expected  to  be  misjudged  on  presentation  at  the 
hospital.  The  obtained  accuracy  of  both  the  computer  and  physician  fall 
within  this  60-70%  expected  error  range.  The  relative  diagnostic  accuracy 
of  the  physician  and  computer  was  dependent  on  disease  area  and  is 
summarized  in  Table  1.  The  computer’s  accuracy  exceeded  that  of  the 
physician  only  for  dyspepsia.  No  cases  were  obtained  in  two  of  the  disease 
areas  evaluated  by  the  computer  program,  diverticulitis  and  perforated 
duodenal  ulcer. 

Table  2  shows  the  disposition  of  the  133  cases  based  upon  the 
diagnostic  accuracies  of  the  computer  and  physicians.  Patient  management 
based  on  computer  diagnosis  approximates  that  of  the  physician.  Physicians 
would  have  appropriately  held,  managed,  and  evacuated  12.4%,  4.7%,  and  7.7% 
more  cases  respectively  than  the  computer. 

Experiment  2 

The  purpose  of  this  study  was  to  evaluate  the  unaided  diagnostic 
accuracy  ~>i  the  corpsnan.  It  was  conducted  in  the  clinical  setting  of  the 
Emergency  Room  of  the  Boone  Clinic,  bittle  Creek,  Virginia. 
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Procedure 


Method 

Students  at  the  Independent  Duty  Technicians  School  (IDT) ,  Health 
Sciences  and  Education  Training  Command,  Portsmouth,  Virginia,  and 
Emergency  Room  Physicians  participated  in  the  study.  The  composition 
and  training  of  the  IDT  students  was  similar  to  those  at  San  Diego  with 
the  exception  that  they  spent  only  one  week  on  their  rotation. The 
students  were  not  instructed  in  the  use  of  the  computer-diagnostic  system 
and  did  not  use  the  computer  data  sheets.  Prior  to  their  rotation,  they 
were  familiarized  with  the  study.  Participating  physicians  were  Physicians 
Assistants,  Interns,  and  General  Duty  Medical  Officers. 

The  patient  population  was  similar  to  that  used  in  San  Diego.  The 
IDT  student  was  the  first  to  see  the  patient.  Subsequent  to  his  examination, 
and  without  consulting  with  a  physician,  he  recorded  his  diagnosis  on  an 
examination  record  form.  The  attending  physician  recorded  his  own  diagnosis 
on  the  same  form.  The  procedure  for  obtaining  the  final  diagnosis  was  similar 
to  that  for  Experiment  1  with  the  following  exceptions :  At  the  end  of 
an  eight  week  period,  all  patients  who  had  not  been  admitted  or  returned 
for  a  subsequent  visit  were  followed  up  with  telephone  contact  to  determine 
whether  there  had  been  any  further  difficulties.  Diagnostic  accuracy  of  the 
corpsman  was  evaluated  by  comparing  his  diagnosis  to  the  final  diagnosis. 
Results 

Seventy-five  cases  of  abdominal  pain  were  obtained  during  the  fifteen 
week  period  between  January  1979  and  May  1979.  Thirty-three  cases  were 
omitted  from  the  analysis  because  of  the  absence  of  a  confirmed  diagnosis. 


missing  data,  or  the  presence  of  a  clinical  problem  inappropriate  to  the 
study. 

No  cases  were  obtained  for  Small  Bowel  Obstruction,  Cholecystitis, 
Diverticulitis  and  Perforated  Duodenal  Ulcer.  This  is  not  surprising 
since  the  sample  of  cases  in  this  study  is  relatively  small  and  these 
diseases  have  a  low  probability  of  occurance.  For  purposes  of  comparison 
with  computer  diagnostic  performance,  diagnoses  of  appendicitis,  dyspepsia 
and  renal  colic  were  combined  into  a  single  "Specific"  category.  Since 
the  computer  was  not  used  in  this  study,  the  data  from  Balboa  was  used 
for  the  comparisons  made  in  Table  3.  The  corpsman  diagnosed  "Non-specific" 
diseases  better  than  the  computer;  the  computer  was  more  accurate  than  the 
corpsman  for  the  "Specific"  diseases. 

Discussion 

deDombal  (1975)  has  reported  diagnostic  accuracies  for  the  computer 
between  81%  and  91%;  the  overall  diagnostic  accuracy  we  obtained  was  only 
61%.  This  difference  may  be  attributable  to  characteristics*  of  the  data 
base,  experience  of  the  corpsmen,  and  training  variables.  The  population 
from  which  the  data  base  was  developed  differs  from  that  to  which  it  was 
applies.  The  data  base  was  intended  for  use  with  the  general  population  of 
patients  which  might  present  with  abdominal  pain  at  Leeds.  The  data  from 
which  the  data  base  was  developed  placed  no  restrictions  upon  age  of  the 
patient  and  was  derived  primarily  from  patients  who  had  been  previously 
seen  by  a  physician  and  were  referred  to  a  hospital.  Our  application 
population  was  restricted  with  an  age  range  between  14  and  60  years^  and 


had  not  previously  been  seen  by  a  physician.  A  data  base  more  appropriate 
for  this  population  is  presently  under  development. 

Although  Navy  Corpsmen  have  intensive  classroom  training,  the  program 
provides  for  relatively  little  clinical  experience  before  going  aboard  a 
submarine.  Studies  currently  being  conducted  in  our  laboratory  indicate 
that  there  may  not  be  sufficient  transfer  from  the  classroom  to  the  clinic 
to  enable  them  to  correctly  diagnose  the  more  serious  evacuable  diseases. 
While  corpsmen  at  San  Diego  were  given  special  training  in  the  elicitation 
of  history  and  physical  symptoms  required  for  the  use  of  the  computer, 
in  the  absence  of  clinical  experience,  the  data  were  probably  not  adequately 
elicited  and  recorded.  The  paramedics  and  physicians  in  deDombal's  studies 
had  either  extensive  experience  in  the  use  of  the  system  or  extensive 
clinical  experience.  Further  evaluations  are  being  undertaken  in  these  areas 

As  shown  by  Table  3,  the  computer's  diagnosis;  was  superior  to  the 
corpsman’ s  for  the  "Specific"  diseases  but  not  for  the  "Non-specific". 

Since  most  of  the  corpsman  at  this  point  in  training  do  not  have  extensive 
clinical  experience,  their  level  of  performance  was  surprising.  One 
possible  explanation  is  that  contrary  to  our  experimental  design,  the 
corpsman  did  not  diagnose  independently  of  the  physician.  However,  the 
physician’s  diagnostic  accuracy  of  61%  in  contrast  to  the  corpsman's 
accuracy  of  29%  for  the  "Specific"  diseases  does  not  support  this  hypothesis. 
The  more  probable  explanation  for  this  difference  may  be  in  the  relative 
probability  of  occurance  of  diseases.  If  the  corpsmen  in  this  study  had 
made  no  other  diagnosis  than  "Non-specific"  pain,  they  would  have  been 
correct  83%  of  the  time.  This  hypothesis  is  strongly  supported  by  the 


overall  diagnostic  accuracy  of  83.3%  (Table  3)  obtained  in  this  study. 

The  high  level  of  diagnostic  accuracy  may  have  been  in  part  a  function  of 
probability  matching  rather  than  of  diagnostic  skill.  We  are  presently 
investigating  this  hypothesis. 

Our  data  suggests  that  the  computer-based  diagnostic  system, 
developed  for  use  by  physicians  in  a  hospital  setting,  can  be  adapted  for 
use  by  paramedics  in  an  isolated  environment.  The  effectiveness  of  the 
computer  in  diagnosing  evacuable  diseases  is  encouraging  and  can  result 
in  the  reduction  of  both  the  loss  of  a  patient  and  serious  complication. 

It  is  not  likely  that  the  present  tendency  of  the  computer  program  to 
place  some  non-evacuable  patients  in  other  categories  would  result  in 
unnecessary  evacuations.  Unless  the  situation  is  acute,  patients  on  board 
a  submarine  are  normally  observed  for  a  period  fo  time  before  being 
evacuated.  As  the  illness  progressed,  the  correct  diagnosis  would 
become  more  obvious  and  evacuation  would  not  occur.  Vie  are ,  however, 
concerned  with  this  problem  and  are  evaluating  methods  of  correcting  it. 

Disclaimer 

The  opinions  expressed  in  this  paper  are  those  of  the  authors  and 
do  not  necessarily  reflect  the  views  of  the  Navy  Department  at  large. 
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FIG.  2  Flow  Chart  of  the  Abdominal  Pain  Program 
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FIG.  3  Datasheet  for  Taking  History 
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FIG.  4  Datasheet  for  Physical  Examination 
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FIG. 5  Summary  of  data  input  to  computer.  Diagnostic  probabilities  are 
in  percent;;.  The  computer  diagnosis  of  appendicitis  was  verified 
at  the  time  of  surgery. 


'  „ 

Disease 

Computer 

Physician 

1 

' 

Non-Specific  Pain  62 

Dyspepsia  55 

Renal  Cholic 
Small  Bowel  Obstruction 

9 

Cholistitis 


Appendicitis 


Table  2 

Patient  Management  Based  Upon  Computer  and  Physician  Diagnostic  Accur 


Disposition 


Computer 


Physician 


76.7  89.1 

18.0  10.3 


Appropriately  Managed 
Inappropriately  Evacuated 
Inappropriately  Held 


5.3 


0.6 


